Mixed bi-subject kinship verification via multi-view multi-task learning

Bi-subject kinship verification addresses the problem of verifying whether there exists some kind of kin relationship (i.e., father–son, father-daughter, mother–son and mother–daughter) between a pair of parent–child subjects based purely on their visual appearance. The task is challenging due to the involvement of two different subjects possibly with different genders and ages. In addition, collecting sufficient training samples for each type of kinship is difficult. In this work, we present a novel method to address these issues by considering each type of kin relation verification as one task and learning them at one time in the framework of multi-task learning, by sharing feature sets and useful structures among the tasks. Particularly our contributions are three folds: first, we introduce a new type of learning problem, called mixed bi-subject kinship verification, to the topic of bi-subject kinship verification: instead of simply verifying whether some fixed kinship relationship (e.g., mother–son) can be established for a given pair of parent–child images, we try to figure out whether any type of the four kinship relations can be established according to the visual features of the image pair, with no need to know the genders of the subjects to be verified beforehand. Second, we propose a novel multi-task learning method to address this problem with two transformation matrices – one is shared amongst all the tasks and the other is unique to each task. Both matrices are simultaneously learned in a joint framework, which enables our algorithm to utilize the common knowledge of the four tasks. Third, we propose a multi-view multi-task learning(MMTL) method to perform multiple feature fusion to improve the mixed bi-subject kinship verification performance. Extensive experiments on the large scale KinFaceW kinship database demonstrate the feasibility and effectiveness of the proposed algorithm. & 2016 Elsevier B.V. All rights reserved.